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Abstract. The surfaces of antimony are characterized by the presence of spin-split 
states within the projected bulk band gap and the Fermi contour is thus expected 
to exhibit a spin texture. Using spin-resolved density functional theory calculations, 
we determine the spin polarization of the surface bands of Sb(110). The existence of 
the unconventional spin texture is corroborated by the investigations of the electron 
scattering on this surface. The charge interference patterns formed around single 
scattering impurities, imaged by scanning tunneling microscopy, reveal the absence of 
direct backscattering signal. We identify the allowed scattering vectors and analyze 
their bias evolution in relation to the surface-state dispersion. 



Unconventional spin texture of a topologically nontrivial semimetal Sb(llO) 2 

Pure bismuth and antimony are group-V semimetals and their surfaces support 
spin-orbit split surface states existing within the projected bulk band gap [I]- [5]. The 
two materials differ however, with respect to the topological character of their bulk 
bands. Whereas pure bismuth is topologically trivial, adding a small amount of Sb 
into Bi is enough to drive the system into a topological insulator phase [6j [7j. The 
spin-nondegenerate surface states of the topological insulator Bi(!_ x )Sb x (0.09<x<0.18) 
continuously connect the valence and the conduction band and their existence is derived 
directly from the fundamental considerations of the parity characteristics of the bulk 
bands [8]. As more Sb is added into the alloy, the system becomes semimetallic again, 
it keeps however the nontrivial topological order of the bulk bands [7J. Thus, the 
other extreme case of the Bi(!_ x )Sb x alloy, pure antimony, is a semimetal with bulk 
characteristic of a strong topological insulator. 

The (111) surfaces of the topologically nontrivial materials, including the 
Bi(i_ x )Sb x alloy and Sb, have been extensively investigated by spin and angle resolved 
photoemission spectroscopy (ARPES) and scanning tunneling microscopy (STM) [4]- 
[7j,[9]-[l2]. These studies allowed to determine the spin textures [3 El ED], map 
the surface band dispersion [lj-[7j and revealed an unconventional electron dynamics 
[HJ CE21 [13] . At the same time, studies addressing non-(lll) surfaces of those materials 
are almost nonexistent. Only recently, we have shown that Sb(llO) exhibits spin-split 
surface states but, contrary to predictions based on its bulk topology, their dispersion 
properties are those of a trivial material . This is due to the semimetallic character 
of the surface, which relaxes the character of the surface state bands imposed by the 
bulk topology [3]. The spin polarization of these surface bands remains, however, still 
unknown. Since Antimony has a weaker spin-orbit coupling than bismuth, it is crucial 
to prove if a sufficient degree of spin polarization still remains in the split surface bands 
and how it affects its electron scattering properties. 

The spin texture of the surface bands can be addressed indirectly by investigating 
the electron scattering processes [T2].[T3]-[T?]. The spin-orbit coupling affects the 
electron scattering probabilities, making them dependent on the overlap of the spin 
wavefunctions of the final and initial scattering state. In particular, the characteristic 
feature of spin-orbit coupled systems is the suppression of direct backscattering, because 
the states with opposite momenta are required to have orthogonal spins by time reversal 
symmetry [HI [T5] . The modulation in the charge local density of states (LDOS) 
which arises due to the interference of the electrons scattering at surface impurities 
is directly accessible with scanning tunneling microscopy and spectroscopy (STS). The 
scattering vectors q ) connecting the initial and final states in the reciprocal space, can 
be identified by examining the Fourier transform (FT) of the interference patterns [19J. 
By demonstrating the lack of backscattering signal, this method allowed to confirm 
the existence of the spin texture at the surfaces of semimetallic bismuth and antimony 
P~3j [HI, E] as we U as topological insulators Bi(i_ x )Sb x , Bi 2 Se3 and Bi 2 Te3 [T2l [T5l [T6] . 

In this paper, we investigate the electron scattering processes on a non-(lll) surface 
of topologically nontrivial material, Sb(llO), in order to gain experimental insight 
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into the spin texture of its surface electronic bands. We observe strong modulation 
in the LDOS around single adatoms on the surface and correlate the features in 
the Fourier transformed images to the specific scattering vectors. We identify those 
scattering events by analyzing the Fermi contour measured by ARPES and the spin 
polarization of the surface bands predicted by relativistic density functional theory 
(DFT) calculations. The resulting spin texture is highly anisotropic and strongly varies 
with the character of the bands, deviating from the conventional picture of a Rashba 
surface. The experimental STM data do not show any direct backscattering between 
the nondegenerate surface bands, confirming their spin-split nature and the spin texture 
of the Fermi contour. 

We have investigated the surface electronic structure of Sb(llO) using STM and 
ARPES. The STM exeriments were performed in a custom-made microscope working 
in an ultrahigh vacuum at 5K. The ARPES data were taken at the SGM-III beamline 
of the synchrotron radiation facility ASTRID in Aarhus [20]. The combined energy 
and angular resolution were better than 10 meV and 0.13°, respectively. The data 
shown here were all taken using a photon energy of 20 eV. During the photoemission 
measurements the sample was kept at a temperature of 60 K. The Sb(110) single crystal 
surface was cleaned insitu by repeated sputtering and annealing cycles. 

We modeled the electronic and spin structure of the Sb(110) using a non-collinear 
DFT [2T1 [22] within the Perdew-Burke-Ernzerhof implementation [23] of the generalized 
gradient approximation. We considered a repeated slab system consisting of 54 layers 
relaxed up to forces < 10 -4 Ry/a.u. and used fully relativistic norm-conserving 
pseudopotentials as described in Ref. [21] with the energy cutoff corresponding to 
E c =60 Ry. In order to calculate the projection of the bulk band structure onto the 

(110) surface, we have also used the tight-binding scheme of Liu and Allen [24J. 

The Sb(110) surface shows several spin-orbit split surface states within the 
projected bulk band gap. In Figs.[j]and[2] we present electronic structure of Sb(110) as 
measured by ARPES and calculated by DFT. The dispersion of the surface bands and 
their topological character were discussed in detail in Ref. [3j. Here, we summarize the 
most important properties of this surface, which directly affect the electron scattering 
processes, i.e. the dispersion of the surface bands in the vicinity the Fermi level and their 
spin texture. The electronic structure of Sb(110) along certain high symmetry directions 
is shown in Fig.[j](a) and the experimentally determined Fermi contour is presented in 
Fig.[2](c) (grey-scale background). For clarity, the lines have been superimposed on the 
data in Fig.[2](c), showing schematically the shape of the contour. The most important 
spectroscopic features which we identify in both calculations and the experimental data 
are: (i) the hole pocket around the M point, giving rise to circular contour in Fig.[2](c), 
(ii) a pocket around Xl, identified in the Fermi contour as a 'butterfly' shaped feature; 

(111) an extended hole pocket around f , observed as a weak feature in ARPES data. 
In addition, ARPES reveals a very shallow electron pocket (dotted lines in Fig.[j](a) 
in X 2 M and YM directions), which is not well reproduced by the calculations. This 
pocket is particularly important to understand the electron scattering dynamics in the 
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Figure 1. a) Surface state dispersion of Sb(llO) measured by ARPES and calculated 
by DFT. The ARPES intensity is coded in the grey scale; the continuous lines 
correspond to the calculated surface states. Yellow shading depicts the projected bulk 
states. The dashed line highlights the shallow electron pockets revealed by ARPES, 
but missed in DFT. (b) dl/dV spectrum of a clean Sb(llO) surface, showing the LDOS 
in the bias range of the surface states. The features in the spectrum correspond to the 
onset of the surface bands. The inset image shows the atomically resolved topography 
of the clean surface. 




Figure 2. Calculated spin texture within first surface Brillouin zone, for the lower (a) 
and upper (b) subband. The color code represents the degree of the spin polarization 
II (see text for details), arrows show direction of the spin in the surface plane, (c) 
Fermi contour of Sb(llO) measured by ARPES. The continuous lines are the schematic 
representation of the ARPES data; the dotted line represents parts of the contour with 
vanishing ARPES intensity. The arrows schematically show the spin polarization of 
the contour, as deduced from plots in (a) and (b). 
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vicinity of the Fermi level, since it contributes strongly to the density of states in this 
energy window [14]. In fact, the experimentally measured dl/dV spectrum (Fig.[j](b)), 
which reflects the LDOS of a clean Sb(llO) surface, shows a peak in the vicinity of 
the Fermi level, marking the onset of the electron pocket. Several other sharp features 
in the dl/dV spectrum correspond to the onset (negative bias) and the top (positive 
bias) of the surface bands. Due to the relatively weak spin-orbit coupling in Sb, the 
splitting of the bands is rather small so that in certain parts of the Brillouin zone the 
two spin partners are close to being degenerate. Accordingly, some of the features of the 
Fermi contour in Fig.[2](c), the high intensity feature along YX\ direction and crossing 
along X\M direction, are double crossings and the two states with opposite spin become 
almost degenerate. 

To determine the spin of the Fermi contour we calculated the spin texture of the 
upper and lower subband within the first Brillouin zone, as shown in Fig,[2] (a) a nd 
(b), respectively. The degree of certainty about the electron spin, II = \Jj2i(Si) 2 , is 
represented by the color scale in the background (II G [0, 1/2]). The direction of the 
spin in the surface plane is depicted by arrows. The polarization out of plane becomes 
important only for the lower subband, close to the X 2 point. Across the rest of the 
Brillouin zone, the expectation value of the polarization out of plane remains very small. 
The spin texture of any constant energy contour can be now defined by considering 
separately the spin polarization for the lower and upper subband. In Fig.[2](c) the spin 
polarization deduced from plots in (a) and (b) is superimposed on the Fermi contour of 
Sb(110). We find that the spin direction is almost constant as we move along one side of 
the butterfly-shaped pocket and rotates along the M pockets. The features along YX\ 
and X\M correspond to almost double crossings and, consequently, have contributions 
from both spin-split bands, and thus, two opposite spins. 

The spin texture of the constant energy contours will affect the scattering of the 
electrons by the surface impurities. In particular, the systems with spin-orbit split 
surface states are characterized by the absence of direct backscattering, imposed by 
the time reversal symmetry [18] . In order to experimentally verify the spin texture of 
Sb(110) and identify the allowed scattering vectors we analyzed the charge interference 
patterns formed by electrons scattering on a single adatom (Fig. [3]). In the bias range 
between -80 and 240 meV we observe pronounced modulation in the LDOS signal. A 
series of dl/dV spectroscopic images in Fig.[3](b)-(g) shows the evolution of the patterns 
for several chosen bias voltages. The Fast Fourier Transform (FFT) of the images ((i)- 
(n)) shows high intensity features, corresponding to the allowed scattering vectors if. 

Close to the Fermi level ((i) and (j)) the most distinctive features of the FFT [25j 
are two double arc-shaped features along YX\ direction (labeled q\ and q 2 ) and the four 
spots lying symmetrically around the center (labelled g 3 ). The features disperse with 
increasing sample bias as shown in more detail in Fig.[4](a)-(c). Each graph represents 
a set of the linescans through the FFT images taken at different sample bias in certain 
high symmetry direction (FXi, YM and YX 2) respectively). The vectors q\ and q 2 
get smaller until at around 100 meV the feature q 2 vanishes from the FFT and the q\ 
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Figure 3. STM imaging of the interference patterns formed around a single impurity 
atom on Sb(110). The adatoms (presumably Sb) have been obtained by control 
tip indentation in the substrate, (a) Constant current topography image of the 
impurity (1=0.5 nA, V=60 mV). (b)-(g) Energy resolved dl/dV images showing electron 
interference patterns, at bias voltages 30 mV, 60 mV, 80 mV, 100 mV, 130 mV and 
160 mV, respectively. A corresponding FFT image is shown below each real space 
image: (h)-(n). The full FFT of the topography image (h) shows the main crystal 
directions and bright points corresponding to the atomic corrugation. The images (i) - 
(n) have been cut out of the full FFT (dotted square in (h)) for a more detailed view 
of the scattering vectors. The scattering vectors referred to in the text are marked as 




Figure 4. Dispersion of the scattering vectors in the directions YX\ (a), f M (b) and 
YX2 (c). The non-dispersive points correspond to the the atomic corrugation spots in 
the FFT. The labelling of the scattering vectors is the same as in Fig. [3] 



transform into a cicular shape. The four features q% move to the center of the FFT and 
disappear also around at 100 meV. At the same bias range two more scattering events 
appear: the features in the direction TX 2 and g 5 in the direction TXi. 

In order to establish the origin of the features observed in FFT, we consider the 
shape and spin texture of the constant energy contour. In the simple picture, the 
scattering probability P(<f, eV) at energy eV between the k -states separated by the 
scattering vector ~tf , is determined by the electron density of states p of the final and 



Unconventional spin texture of a topologically nontrivial semimetal Sb(llO) 



the initial state, weighted by the overlap of the spinor wavefunctions (p$ [12] : 



P(q,eV) = J p(k, eV)p(k + q,eV)(^ s (k)\^s(k + q})d 2 k 



(1) 



Thus, the spin dependent part of the scattering amplitude will be maximal if the initial 
and final spins are parallel and zero when the two spins are antipararell. 

If the spin contribution to the scattering amplitudes is neglected, the map 
of the scattering vectors allowed by the Fermi contour can be evaluated directly 
from the photoemission data, by calculating the convolution of the ARPES spectra. 
The autocorrelation image (AC- ARPES) can be derived from the ARPES intensities 



The AC- ARPES defined in this way reproduces the features of the joint density 
of states (JDOS), which in turn maps out the LDOS contribution to the scattering 
probabilities. While the quantitative comparison between AC-ARPES and JDOS is 
not straightforward, since the ARPES intensities I are related to the electron spectral 
function through an energy and fc-dependent matrix element, the autocorrelation image 
stills allows for a qualitative analysis of the scattering vectors [26j [27] . For spin 
degenerate Fermi contour, the AC-ARPES can be directly compared to the FFT of 
the interference patterns resolved by STM at low bias [27] . 

We have calculated the AC-ARPES map for Sb(llO) (Fig.[5](a)), neglecting its 
spin texture, and compared it to the FT-STM map measured close to the Fermi 
level (Fig.[5](d)). We find that certain parts of the AC-ARPES are suppressed in 
the STM data. In fact, many of the features present in AC-ARPES but absent in 
the FFT map can be identified as direct backscattering events, which are expected 
to be prohibited by the spin texture of Sb(llO). Thus, to describe the experimental 
STM data more accurately, we introduce into the cross-correlation integral (Eq. (2)) an 
additional term: 1 — cos[Z( k , k + if)]. While this basic term does not account for the 
anisotropic spin texture of Sb(llO), it suppresses the backscattering in a simple way. 
The result (Fig.[5](b)) reproduces the FFT map to a greater degree. Large parts of the 
original AC-ARPES map in Fig.[5](b) are now diminished and the remaining features 
correspond relatively well to the main spots of the FFT. We can thus conclude that the 
backscattering is indeed absent on Sb(llO) surface, confirming the spin-split character 
of its surface bands. 

We can now associate the spots resolved in the FT-STM map (Fig.[5fd)) with the 
specific scattering vectors. First, we identify the feature q\ as interband scattering 
between the edges of the 'butterfly wings' in TXi direction (Fig.[5](c)). This feature 
of the Fermi contour corresponds to a very shallow and extended electron pocket and 
is thus expected to have a high DOS and contribute strongly to the scattering at this 
energy range. With increasing bias, q± disperses very steeply (see Fig.[I](a)), reflecting 
the dispersion of the left band which forms the electron pocket in TM direction. In 
principle, the scattering between the 'butterfly wings' should also give rise to features in 



J(fc, eV), as [261E7]: 
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FFT in the TX 2 direction. They are indeed resolved in the AC-ARPES images Fig.[5](a) 
but we observe them only as faint spots in FT-STM image, possibly due to partial 
suppression by the overlap of the spinor wavefunctions. Next, we assign the feature q 2 
to the backscattering between the TXi-crossing points. At first glance, such scattering 
event seems to violate the time-reversal symmetry. However, at this point of the Fermi 
contour the two spin-split bands are close to be degenerate. Therefore both spin partners 
are present, allowing the backscattering process, in a similar way as it happens on a 
Au(lll) surface [T8] . Since backscattering is a preferred scattering process on metal 
surfaces, this feature has a rather high intensity in the FFT [18]. The AC-ARPES map 
in Fig.|i](b) does not reproduce it though, as the 1 — cos[Z( k , k + if)] term included in 
the integral excludes all the backscattering processes. This assignment of q 2 is further 
confirmed by the bias evolution of this feature (Fig.[I](a)). As the energy increases, the 
bands which cross the Fermi level along TXi direction disperse towards the T point. 
Thus, the feature q 2 moves towards the center of the FT-STM image (Fig.[3](a)), until 
the top of the band is reached and it disappears from the FFT. This point at ~ lOOmeV 
is not accurately reproduced by the band structure calculations in Fig. la (~ 190meV), 
what we attribute to the inability of DFT to reproduce all fine structure details in the 
meV range. The spots marked as q% result from the forward scattering between the edges 
of the butterfly wing (the electron pocket) and the f Xt-crossing point. The analysis of 
the band's dispersion also shows that these features should move toward the center of 
the FFT image (Fig.gc)). 

The identification of the scattering events far above the Fermi level (q^ and q$) is 
not as clear, because of the uncertainties in the calculated dispersion and the absence 
of photoemission data. We can tentatively explain their origin by the appearance of the 
new high DOS feature (peak at 100 meV in Fig.[l](b)), when the sample bias reaches the 
top of the bands near the M and f point. 

It is interesting to compare these results to the previous study of electron scattering 
on Bi(110), another extreme case of the BiSb alloy. In Bi(110) the pronounced spin- 
splitting of the surface bands leads to a clear manifestation of the spin-dependent 
scattering process. In fact, only spin conserving scattering events are observed in the 
Fourier transformed dl/dV maps on Bi(110) [T4l [T7] . 

Antimony has a smaller spin-orbit coupling and the spin-split of the bands is 
generally smaller [3]. Although, the bands maintain a certain spin texture (as evidenced 
in Fig. 2), the role of the spin in the formation of the interference patterns is 
experimentally less evident, and certain backscattering events could be identified. These 
are caused because at certain points of the Fermi surface with high DOS, the two 
sub-bands are so close in K-space that scattering between them will resemble the 
backscattering events in the interference patterns, in a similar way as described in Ref. 

m 

In summary, we have predicted theoretically and confirmed experimentally the 
unconventional spin texture of the Sb(110) surface. In spite of a smaller spin-orbit 
coupling, the spin polarization of the surface bands still dominates the electron dynamics 
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Figure 5. (a) The autocorrelation of the ARPES data of the Fermi contour of Sb(llO), 
neglecting its spin texture, (b) The autocorrelation signal as in (a), but with suppressed 
backscattering (see text for details), (c) The Fermi contour of Sb(llO) with identified 
scattering vectors, (d) FFT measureded at bias close to the Fermi level (30mV). 

of this surface. The LDOS interference patterns formed around single impurities reveal 
the dominance of non-direct backscattering events as in the related compound Bi(llO). 
The identification of the scattering events was done combining ARPES measurements at 
E F with the state's dispersion above E^, measured from energy dependent differential 
conductance maps, and interpreted on the basis of DFT calculations of the spin texture 
and dispersion of the surface state bands. The smaller spin-orbit interaction of Sb(llO) is 
reflected in a smaller band splitting what, in some cases, give rise to interband scattering 
with wavevectors close to the pure backscattering case. 
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